{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "49a1a69c-8605-47b1-842b-005b8f0e89ec",
   "metadata": {},
   "outputs": [],
   "source": [
    "#| default_exp m5"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6c3dbb4c-07aa-4421-a8f1-3cc36f4c9777",
   "metadata": {},
   "source": [
    "# M5 dataset\n",
    "\n",
    "> Download and evaluate the M5 dataset.\n",
    "\n",
    "The 2020 M5 competition was organized by the Makridakis Open Forecasting Center was organized by University of Nicosia and hosted in Kaggle. The main task consisted in providing hierarchical point and probabilistic predictions for 28 days ahead for the sales of Walmart stores in three US States (California, Texas, and Wisconsin). The dataset is organized at the item, department, product and store level details. Additionally it contains temporal covariates like price, promotions, special event and calendar variables.\n",
    "\n",
    "\n",
    "[University of Nicosia. The M5 competition: Estimate the unit sales of Walmart retail goods. Kaggle Competition, 2020.](https://www.kaggle.com/c/m5-forecasting-accuracy/)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e1c7e36a-58cf-44c5-9638-c1aa3c108cc0",
   "metadata": {},
   "outputs": [],
   "source": [
    "#| hide\n",
    "from nbdev import *\n",
    "from fastcore.test import test_close\n",
    "%load_ext autoreload\n",
    "%autoreload 2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c4c83ec4-30a2-4229-9d54-b836f762f5ae",
   "metadata": {},
   "outputs": [],
   "source": [
    "#| export\n",
    "import os\n",
    "from dataclasses import dataclass\n",
    "from typing import Optional, Tuple, Union\n",
    "\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "\n",
    "from datasetsforecast.utils import download_file"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e30abbd7-91b7-43af-904c-975445af26d4",
   "metadata": {},
   "source": [
    "## Download data class"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ede0cfa5-fe76-43fb-b835-cefc69ececd0",
   "metadata": {},
   "outputs": [],
   "source": [
    "#| export\n",
    "@dataclass\n",
    "class M5:\n",
    "    \n",
    "    # original data available from Kaggle directly\n",
    "    # pip install kaggle --upgrade\n",
    "    # kaggle competitions download -c m5-forecasting-accuracy\n",
    "    source_url: str = 'https://github.com/Nixtla/m5-forecasts/raw/main/datasets/m5.zip'\n",
    "    \n",
    "    @staticmethod\n",
    "    def download(directory: str) -> None:\n",
    "        \"\"\"\n",
    "        Downloads M5 Competition Dataset.\n",
    "        \n",
    "        Parameters\n",
    "        ----------\n",
    "        directory: str\n",
    "            Directory path to download dataset.\n",
    "        \"\"\"\n",
    "        path = f'{directory}/m5/datasets'\n",
    "        if not os.path.exists(path):\n",
    "            download_file(directory=path,\n",
    "                          source_url=M5.source_url,\n",
    "                          decompress=True)\n",
    "            \n",
    "    @staticmethod\n",
    "    def load(directory: str, cache: bool = True) -> Tuple[pd.DataFrame, \n",
    "                                                          pd.DataFrame, \n",
    "                                                          pd.DataFrame]:\n",
    "        \"\"\"Downloads and loads M5 data.\n",
    "\n",
    "        Parameters\n",
    "        ----------\n",
    "        directory: str\n",
    "            Directory where data will be downloaded.\n",
    "        cache: bool\n",
    "            If `True` saves and loads.\n",
    "            \n",
    "        Returns\n",
    "        -------\n",
    "        Y_df: pd.DataFrame\n",
    "            Target time series with columns ['unique_id', 'ds', 'y'].\n",
    "        X_df: pd.DataFrame\n",
    "            Exogenous time series with columns ['unique_id', 'ds', 'y'].\n",
    "        S_df: pd.DataFrame\n",
    "            Static exogenous variables with columns ['unique_id', 'ds']. \n",
    "            and static variables. \n",
    "        \"\"\"\n",
    "        path = f'{directory}/m5/datasets'\n",
    "        file_cache = f'{path}/m5.p'\n",
    "        \n",
    "        if os.path.exists(file_cache) and cache:\n",
    "            Y_df, X_df, S_df = pd.read_pickle(file_cache)\n",
    "            \n",
    "            return Y_df, X_df, S_df\n",
    "        \n",
    "        M5.download(directory)\n",
    "        # Calendar data\n",
    "        cal_dtypes = {\n",
    "            'wm_yr_wk': np.uint16,\n",
    "            'event_name_1': 'category',\n",
    "            'event_type_1': 'category',\n",
    "            'event_name_2': 'category',\n",
    "            'event_type_2': 'category',\n",
    "            'snap_CA': np.uint8,\n",
    "            'snap_TX': np.uint8,\n",
    "            'snap_WI': np.uint8,\n",
    "        }\n",
    "        cal = pd.read_csv(f'{path}/calendar.csv', \n",
    "                          dtype=cal_dtypes, \n",
    "                          usecols=list(cal_dtypes.keys()) + ['date'], \n",
    "                          parse_dates=['date'])\n",
    "        cal['d'] = np.arange(cal.shape[0]) + 1\n",
    "        cal['d'] = 'd_' + cal['d'].astype('str')\n",
    "        cal['d'] = cal['d'].astype('category')\n",
    "        \n",
    "        event_cols = [k for k in cal_dtypes if k.startswith('event')]\n",
    "        for col in event_cols:\n",
    "            cal[col] = cal[col].cat.add_categories('nan').fillna('nan')\n",
    "        \n",
    "        # Prices\n",
    "        prices_dtypes = {\n",
    "            'store_id': 'category',\n",
    "            'item_id': 'category',\n",
    "            'wm_yr_wk': np.uint16,\n",
    "            'sell_price': np.float32\n",
    "        }\n",
    "\n",
    "        prices = pd.read_csv(f'{path}/sell_prices.csv', \n",
    "                             dtype=prices_dtypes)\n",
    "        \n",
    "        # Sales\n",
    "        sales_dtypes = {\n",
    "            'item_id': prices.item_id.dtype,\n",
    "            'dept_id': 'category',\n",
    "            'cat_id': 'category',\n",
    "            'store_id': 'category',\n",
    "            'state_id': 'category',\n",
    "            **{f'd_{i+1}': np.float32 for i in range(1969)}\n",
    "        }\n",
    "        # Reading train and test sets\n",
    "        sales_train = pd.read_csv(f'{path}/sales_train_evaluation.csv', dtype=sales_dtypes)\n",
    "        sales_test = pd.read_csv(f'{path}/sales_test_evaluation.csv', dtype=sales_dtypes)\n",
    "        sales = sales_train.merge(sales_test, how='left', \n",
    "                                  on=['item_id', 'dept_id', 'cat_id', 'store_id', 'state_id'])\n",
    "        sales['id'] = sales['item_id'].astype(str) + '_' + sales['store_id'].astype(str)\n",
    "        sales['id'] = sales['id'].astype('category')\n",
    "        # Long format\n",
    "        long = sales.melt(id_vars=['id', 'item_id', 'dept_id', 'cat_id', 'store_id', 'state_id'], \n",
    "                          var_name='d', value_name='y')\n",
    "        # paste dates\n",
    "        long['d'] = long['d'].astype(cal.d.dtype)\n",
    "        long = long.merge(cal, on=['d'])\n",
    "\n",
    "        # remove leading zeros from series\n",
    "        long = long.sort_values(['id', 'date'])\n",
    "        without_leading_zeros = long['y'].gt(0).groupby(long['id']).transform('cummax')\n",
    "        long = long[without_leading_zeros]\n",
    "\n",
    "        # prices\n",
    "        long = long.merge(prices, on=['store_id', 'item_id', 'wm_yr_wk'])\n",
    "        long = long.drop(columns=['d', 'wm_yr_wk'])\n",
    "\n",
    "        long = long.rename(columns={'id': 'unique_id', 'date': 'ds'})\n",
    "        Y_df = long[['unique_id', 'ds', 'y']]\n",
    "        cats = ['item_id', 'dept_id', 'cat_id', 'store_id', 'state_id']\n",
    "        S_df = long[['unique_id'] + cats].groupby('unique_id', observed=True).head(1)\n",
    "        x_cols = long.columns.drop(['y'] + cats)\n",
    "        X_df = long[x_cols]\n",
    "        \n",
    "        if cache:\n",
    "            pd.to_pickle((Y_df, X_df, S_df), file_cache)\n",
    "\n",
    "        return Y_df, X_df, S_df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f11ef7c6-4878-48e8-bc88-a17b23a91d77",
   "metadata": {},
   "outputs": [],
   "source": [
    "#| hide\n",
    "Y_df, X_df, S_df = M5.load('./data')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4f409f99-0306-4a71-bbb5-c4ce19a273e2",
   "metadata": {},
   "source": [
    "## Test number of series"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3920d220-646d-410a-be2c-010b0e7ec089",
   "metadata": {},
   "outputs": [],
   "source": [
    "#| hide\n",
    "n_series = 30_490\n",
    "assert Y_df['unique_id'].unique().size == n_series\n",
    "assert X_df['unique_id'].unique().size == n_series\n",
    "assert S_df.shape[0] == 30_490"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7a576b88-13e5-485d-8696-a331e0b6909b",
   "metadata": {},
   "outputs": [],
   "source": [
    "#| hide\n",
    "Y_df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "98e3e6a0-3aef-46ff-aafe-ae016dd030ef",
   "metadata": {},
   "outputs": [],
   "source": [
    "#| hide\n",
    "X_df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7326a877-309b-4cf7-8b8c-1d07fbd2ccf9",
   "metadata": {},
   "outputs": [],
   "source": [
    "#| hide\n",
    "S_df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "71a715f2-2879-4663-b164-8eb28015929b",
   "metadata": {},
   "source": [
    "## Evaluation class"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cebfe34e-4174-48f1-8dee-459281c8846d",
   "metadata": {},
   "outputs": [],
   "source": [
    "#| export\n",
    "class M5Evaluation:\n",
    "    \n",
    "    levels: dict =  dict(\n",
    "        Level1=['total'],\n",
    "        Level2=['state_id'],\n",
    "        Level3=['store_id'],\n",
    "        Level4=['cat_id'],\n",
    "        Level5=['dept_id'],\n",
    "        Level6=['state_id', 'cat_id'],\n",
    "        Level7=['state_id', 'dept_id'],\n",
    "        Level8=['store_id', 'cat_id'],\n",
    "        Level9=['store_id', 'dept_id'],\n",
    "        Level10=['item_id'],\n",
    "        Level11=['state_id', 'item_id'],\n",
    "        Level12=['item_id', 'store_id']\n",
    "    )\n",
    "    \n",
    "    @staticmethod\n",
    "    def load_benchmark(directory: str,\n",
    "                       source_url: Optional[str] = None,\n",
    "                       validation: bool = False) -> np.ndarray:\n",
    "        \"\"\"Downloads and loads a bechmark forecasts.\n",
    "        \n",
    "        Parameters\n",
    "        ----------\n",
    "        directory: str\n",
    "            Directory where data will be downloaded.\n",
    "        source_url: str, optional\n",
    "            Optional benchmark url obtained from \n",
    "            https://github.com/Nixtla/m5-forecasts/tree/master/forecasts.\n",
    "            If `None` returns the M5 winner. \n",
    "        validation: bool\n",
    "            Wheter return validation forecasts.\n",
    "            Default False, return test forecasts.\n",
    "        \n",
    "        Returns\n",
    "        -------\n",
    "        benchmark: numpy array\n",
    "            Numpy array of shape (n_series, horizon).\n",
    "        \"\"\"\n",
    "        path = f'{directory}/m5/datasets'\n",
    "        if source_url is not None:\n",
    "            filename = source_url.split('/')[-1].replace('.rar', '.csv')\n",
    "            filepath = f'{path}/{filename}'\n",
    "            if not os.path.exists(filepath):\n",
    "                download_file(path, source_url, decompress=True)\n",
    "            \n",
    "        else:\n",
    "            source_url = 'https://github.com/Nixtla/m5-forecasts/raw/main/forecasts/0001 YJ_STU.zip'\n",
    "            return M5Evaluation.load_benchmark(directory, source_url, validation)\n",
    "        \n",
    "        benchmark = pd.read_csv(filepath)\n",
    "        mask = benchmark['id'].str.endswith('validation') \n",
    "        if validation:\n",
    "            benchmark = benchmark[mask]\n",
    "            benchmark['id'] = benchmark['id'].str.replace('_validation', '')\n",
    "        else:\n",
    "            benchmark = benchmark[~mask]\n",
    "            benchmark['id'] = benchmark['id'].str.replace('_evaluation', '')\n",
    "            \n",
    "        benchmark = benchmark.sort_values('id', ignore_index=True)\n",
    "        benchmark.rename(columns={'id': 'unique_id'}, inplace=True)\n",
    "        *_, s_df = M5.load(directory)\n",
    "        benchmark = benchmark.merge(s_df, how='left',\n",
    "                                    on=['unique_id'])\n",
    "        \n",
    "        return benchmark\n",
    "    \n",
    "    @staticmethod\n",
    "    def aggregate_levels(y_hat: pd.DataFrame, \n",
    "                         categories: pd.DataFrame = None) -> pd.DataFrame:\n",
    "        \"\"\"\n",
    "        Aggregates the 30_480 series to get 42_840.\n",
    "        \n",
    "        Parameters\n",
    "        ----------\n",
    "        y_hat: pd.DataFrame\n",
    "            Forecasts as wide pandas dataframe with columns ['unique_id'].\n",
    "        categories: pd.DataFrame\n",
    "            Categories of M5 dataset (not used).\n",
    "\n",
    "        Returns\n",
    "        -------\n",
    "        df_agg: pd.DataFrame\n",
    "            Aggregated forecasts as wide pandas dataframe with columns ['unique_id'].\n",
    "        \"\"\"\n",
    "        y_hat_cat = y_hat.assign(total='Total')\n",
    "\n",
    "        df_agg = []\n",
    "        for level, agg in M5Evaluation.levels.items():\n",
    "            df = y_hat_cat.groupby(agg).sum(numeric_only=True).reset_index()\n",
    "            renamer = dict(zip(agg, ['Agg_Level_1', 'Agg_Level_2']))\n",
    "            df.rename(columns=renamer, inplace=True)\n",
    "            df.insert(0, 'Level_id', level)\n",
    "            df_agg.append(df)\n",
    "        df_agg = pd.concat(df_agg)\n",
    "        df_agg = df_agg.fillna('X')\n",
    "        df_agg = df_agg.set_index(['Level_id', 'Agg_Level_1', 'Agg_Level_2'])\n",
    "        df_agg.columns = [f'd_{i+1}' for i in range(df_agg.shape[1])]\n",
    "\n",
    "        return df_agg\n",
    "    \n",
    "    @staticmethod\n",
    "    def evaluate(directory: str, \n",
    "                 y_hat: Union[pd.DataFrame, str],\n",
    "                 validation: bool = False) -> pd.DataFrame:\n",
    "        \"\"\"Evaluates y_hat according to M4 methodology.\n",
    "        \n",
    "        Parameters\n",
    "        ----------\n",
    "        directory: str\n",
    "            Directory where data will be downloaded.\n",
    "        validation: bool\n",
    "            Wheter perform validation evaluation.\n",
    "            Default False, return test evaluation.\n",
    "        y_hat: pandas datafrae, str\n",
    "            Forecasts as wide pandas dataframe with columns\n",
    "            ['unique_id'] and forecasts or\n",
    "            benchmark url from\n",
    "            https://github.com/Nixtla/m5-forecasts/tree/main/forecasts.\n",
    "            \n",
    "        Returns\n",
    "        -------\n",
    "        evaluation: pandas dataframe\n",
    "            DataFrame with columns OWA, SMAPE, MASE\n",
    "            and group as index.\n",
    "        \"\"\"\n",
    "        if isinstance(y_hat, str):\n",
    "            y_hat = M5Evaluation.load_benchmark(directory, y_hat, validation)\n",
    "    \n",
    "        M5.download(directory)\n",
    "        path = f'{directory}/m5/datasets'\n",
    "        if validation:\n",
    "            weights = pd.read_csv(f'{path}/weights_validation.csv')\n",
    "            sales = pd.read_csv(f'{path}/sales_train_validation.csv')\n",
    "            y_test = pd.read_csv(f'{path}/sales_test_validation.csv')\n",
    "        else:\n",
    "            weights = pd.read_csv(f'{path}/weights_evaluation.csv')\n",
    "            sales = pd.read_csv(f'{path}/sales_train_evaluation.csv')\n",
    "            y_test = pd.read_csv(f'{path}/sales_test_evaluation.csv')\n",
    "            \n",
    "        # sales\n",
    "        sales = M5Evaluation.aggregate_levels(sales)\n",
    "        def scale(x):\n",
    "            x = x.values\n",
    "            x = x[np.argmax(x!=0):]\n",
    "            scale = ((x[1:] - x[:-1]) ** 2).mean()\n",
    "            return scale\n",
    "        scales = sales.agg(scale, 1).rename('scale').reset_index()\n",
    "        \n",
    "        # y_test\n",
    "        y_test = M5Evaluation.aggregate_levels(y_test)\n",
    "        \n",
    "        #y_hat\n",
    "        y_hat = M5Evaluation.aggregate_levels(y_hat)\n",
    "        \n",
    "        score = (y_test - y_hat) ** 2\n",
    "        score = score.mean(1)\n",
    "        score = score.rename('rmse').reset_index()\n",
    "        score = score.merge(weights, how='left', \n",
    "                            on=['Level_id', 'Agg_Level_1', 'Agg_Level_2'])\n",
    "        score = score.merge(scales, how='left', \n",
    "                            on=['Level_id', 'Agg_Level_1', 'Agg_Level_2'])\n",
    "        score['wrmsse'] = (score['rmse'] / score['scale']).pow(1 / 2) * score['weight']\n",
    "        score = score.groupby('Level_id')[['wrmsse']].sum()\n",
    "        score = score.loc[M5Evaluation.levels.keys()]\n",
    "        total = score.mean().rename('Total').to_frame().T\n",
    "        score = pd.concat([total, score])\n",
    "        \n",
    "        return score"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f243a243-3b80-4773-9801-00975b53cb34",
   "metadata": {},
   "source": [
    "### URL-based evaluation\n",
    "\n",
    "The method `evaluate` from the class `M5Evaluation` can receive a url of a [submission to the M5 competiton](https://github.com/Nixtla/m5-forecasts/tree/main/forecasts).  \n",
    "\n",
    "The results compared to the on-the-fly evaluation were obtained from the [official evaluation](https://github.com/Mcompetitions/M5-methods/blob/master/Scores%20and%20Ranks.xlsx)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3baf1f52-3b31-48c9-a755-ae01180ce0ca",
   "metadata": {},
   "outputs": [],
   "source": [
    "m5_winner_url = 'https://github.com/Nixtla/m5-forecasts/raw/main/forecasts/0001 YJ_STU.zip'\n",
    "winner_evaluation = M5Evaluation.evaluate('data', m5_winner_url)\n",
    "# Test of the same evaluation as the original one\n",
    "test_close(winner_evaluation.loc['Total'].item(), 0.520, eps=1e-3)\n",
    "winner_evaluation"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "16c7d00c-fda4-4609-a985-523a433e45bc",
   "metadata": {},
   "source": [
    "### Pandas-based evaluation"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1aae1209-943d-438c-a895-56ab2a390e80",
   "metadata": {},
   "source": [
    "Also the method `evaluate` can recevie a pandas DataFrame of forecasts.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "02539952-0ab3-446b-8498-c30b6a786182",
   "metadata": {},
   "outputs": [],
   "source": [
    "m5_second_place_url = 'https://github.com/Nixtla/m5-forecasts/raw/main/forecasts/0002 Matthias.zip'\n",
    "m5_second_place_forecasts = M5Evaluation.load_benchmark('data', m5_second_place_url)\n",
    "second_place_evaluation = M5Evaluation.evaluate('data', m5_second_place_forecasts)\n",
    "# Test of the same evaluation as the original one\n",
    "test_close(second_place_evaluation.loc['Total'].item(), 0.528, eps=1e-3)\n",
    "second_place_evaluation"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4b6faa22-f020-4db2-919c-5d81abcacfbb",
   "metadata": {},
   "source": [
    "By default you can load the winner benchmark using the following."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0b86f46a-de45-4c8c-93a8-47337e978bfd",
   "metadata": {},
   "outputs": [],
   "source": [
    "winner_benchmark = M5Evaluation.load_benchmark('data')\n",
    "winner_evaluation = M5Evaluation.evaluate('data', winner_benchmark)\n",
    "# Test of the same evaluation as the original one\n",
    "test_close(winner_evaluation.loc['Total'].item(), 0.520, eps=1e-3)\n",
    "winner_evaluation"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "33514b7e-8663-46fd-837b-9721c9caaa3a",
   "metadata": {},
   "source": [
    "### Validation evaluation"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "981bbab8-3e21-4731-b5b6-3855e2bb931f",
   "metadata": {},
   "source": [
    "You can also evaluate the official validation set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8ea1dc7d-ae6a-409d-bc0e-ecb72182773b",
   "metadata": {},
   "outputs": [],
   "source": [
    "winner_benchmark_val = M5Evaluation.load_benchmark('data', validation=True)\n",
    "winner_evaluation_val = M5Evaluation.evaluate('data', winner_benchmark_val, validation=True)\n",
    "winner_evaluation_val"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "215ada69-0da2-41a7-92d6-e27bcaac6752",
   "metadata": {},
   "source": [
    "## Kaggle-Competition-M5 References\n",
    "\n",
    "The evaluation metric of the Favorita Kaggle competition was the normalized weighted root mean squared logarithmic error (NWRMSLE).\n",
    "Perishable items have a score weight of 1.25; otherwise, the weight is 1.0.\n",
    "\n",
    "$$ NWRMSLE = \\sqrt{\\frac{\\sum^{n}_{i=1} w_{i}\\left(log(\\hat{y}_{i}+1)  - log(y_{i}+1)\\right)^{2}}{\\sum^{n}_{i=1} w_{i}}}$$\n",
    "\n",
    "Kaggle Competition Forecasting Methods                                                              | 16D ahead NWRMSLE\n",
    ":-------------------------------------------------------------------------------------------------: | :-------: \n",
    "[LGBM](https://www.kaggle.com/shixw125/1st-place-lgb-model-public-0-506-private-0-511/comments) [1] | 0.5091   | \n",
    "[Seq2Seq WaveNet](https://arxiv.org/abs/1803.04037) [2]                                             | 0.5129   |\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "813fbb27-c5f2-44e6-a3a0-8cef208d4c73",
   "metadata": {},
   "source": [
    "1.\t[Corporación Favorita. Corporación favorita grocery sales forecasting. Kaggle Competition Leaderboard, 2018.](https://www.kaggle.com/c/favorita-grocery-sales-forecasting/leaderboard)\n",
    "2.\t[Glib Kechyn, Lucius Yu, Yangguang Zang, and Svyatoslav Kechyn.  Sales forecasting using wavenet within the framework of the Favorita Kaggle competition. Computing Research Repository, abs/1803.04037, 2018](https://arxiv.org/abs/1803.04037)."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "python3",
   "language": "python",
   "name": "python3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
